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FOREWORD 


These  recommendations  are  based  on  lessons  learned  from  developmental  exercises  using 
Distributed  Interactive  Simulation  (DIS)  technologies  to  train  Close  Air  Support  (CAS)  combat 
operations  in  the  Multi-Service  Distributed  Training  Testbed  (MDT2).  They  are  written  to  help 
senior  managers  plan  and  use  DIS  technology  for  prototype  unit  training  applications.  Detailed 
appendixes  referred  to  in  this  report  describe  the  tools  and  methods  used  specifically  for  training 
CAS  and  are  available  from  the  Defense  Technical  Information  Center. 

The  MDT2  program  is  a  complex  of  interrelated  projects  designed  to  develop  and 
evaluate  DIS-based  training  technologies  and  methods  to  meet  Multi-Service  training  needs.  Its 
advocates  and  sponsors  include  (1)  the  Defense  Modeling  and  Simulation  Office  (DMSO), 

(2)  the  Office  of  the  Under  Secretary  of  Defense  (Personnel  and  Readiness)  -  Defense  Manpower 
Data  Center's  (DMDC)  Training  Readiness  Evaluation  and  Analysis  Division  (formerly  DITRA), 
and  (3)  the  four  military  Services.  Thanks  are  extended  to  the  many  Government,  military, 
civilian,  and  contractor  organizations  who  made  the  effort  possible.  The  principal  agencies  and 
laboratories  who  supported  the  project  are: 

-  Director  Defense  Research  and  Engineering/ 

Defense  Modeling  and  Simulation  Office  (DDR&E/DMSO) 

-  Office  of  the  Under  Secretary  of  Defense/ 

Personnel  and  Readiness  (OUSD/P&R) 

-  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI) 

-  U.S.  Army  Training  and  Doctrine  Command/ 

Training  Support  Center 

-  Naval  Air  Warfare  Center 

-  Training  Systems  Division  (NAWCTSD)  -  formerly  NTSC 
—  Aircraft  Division  (NAWCAD) 

-  Marine  Corps  Combat  Development  Command  (MCCDC) 

-  Air  Force  Armstrong  Laboratory/Aircrew  Training 
Research  Division  (AL/HRA) 

-  Trainers  and  Warriors  (Active,  Reserve,  &  National  Guard) 
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Recommendations  for  Planning  and  Conducting  Multi-Service  Tactical  Training  with 
Distributed  Interactive  Simulation  (DIS)  Technology 


1.0  INTRODUCTION 

The  conduct  of  group  or  unit  training  for  combat  historically  involves  military  exercises 
in  field  settings.  Such  exercises  are  costly,  potentially  dangerous,  and  damaging  to  the 
environment.  There  also  are  major  limitations  on  what  can  be  trained  in  field  exercises.  The  use 
of  Distributed  Interactive  Simulation  (DIS)  technology  provides  a  way  for  units  at  different  sites 
to  prepare  for  or  to  supplement  field  training  experience.  This  paper  recommends  practices  for 
planning  and  conducting  tactical  training  using  DIS  technology  with  multi-Service^  groups  even 
though  they  are  geographically  separated. 

DIS  technology  uses  wide  area  communication  networks  to  link  different  simulators  at 
separate  locations.  This  capability  can  be  valuable  for  single-Service  tactical  training  but,  in 
addition,  it  presents  opportunities  for  multi-Service  combat  mission  training.  Many  multi- 
Service  missions  may  benefit  from  such  training— for  example,  air  defense,  tactical  air-to-air 
combat,  combined  amphibious  and  land  assault,  and  fire  support  including  Close  Air  Support 
(CAS).  There  also  are  many  uses  for  operations  other  than  war,  such  as  teams  and  units 
(a)  fighting  large  fires;  (b)  training  disaster  relief,  including  fire  and  rescue,  police,  and  medical 
units;  and  (c)  managing  emergencies  with  multi-disciplinary  units  in  cities  and  at  airports  as  a 
general  case. 

The  Multi-Service  Distributed  Training  Testbed  (MDT2)  program  is  designed  to  develop 
training  opportunities  and  tools  that  will  increase  the  utility  of  multi-Service  training.  The  testbed 
provides  a  realistic,  although  synthetic,  environment  for  training  and  the  flexibility  to  support 
planning,  preparation,  execution,  and  feedback.  Its  first  application  was  to  the  multi-Service 
CAS  mission.  The  recommendations  presented  in  this  paper  are  based  on  the  experience  gained 
from  that  application. 

Both  live  and  virtual  (simulated)  environments  have  training  advantages  which  neither 
alone  can  provide  equally  well  today.  The  live  setting  offers  realistic,  hands-on,  fatiguing 
training,  but  it  rarely  provides  unrestricted  employment  of  weapon  systems.  The  virtual  setting 
lacks  realism  in  other  ways,  because  of  simulator  limitations,  but  it  allows  training  in  activities 
and  procedures  that  are  difficult  or  impossible  to  train  in  live  environments.  In  addition,  a  major 
advantage  of  training  in  virtual  environments  is  that  such  exercises  may  more  easily  be  tailored 
than  live  exercises  to  specific  needs  of  multi-Service  teams  and  groups. 


^Multi-Service  refers  to  more  than  one  military  Service  working  together  for  combat 
operations  but  not  dependent  on  a  Joint  Commander.  Joint  Operations  are  those  conducted  by 
more  than  one  Service  under  a  Joint  Commander. 
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1.1  The  Goal:  Better  Multi-Service  Training 


1.1.1  Purposes  of  MDT2 

The  MDT2  program  is  designed  to  facilitate  DIS-based  training.  The  testbed  begins  the 
process  of  designing  and  assessing  prototype  training  methods  and  tools  to  enhance  distributed 
training.  Recommendations  about  these  prototypes  form  the  major  portion  of  this  paper  and 
provide  a  starting  point  for  addressing  the  following  training  needs: 

(1)  Effective  use  of  distributed  training.  What  skills  are  best  trained  (and  how  well)  using 
DIS?  How  can  DIS  technology  best  be  incorporated  into  a  program  for  combat  mission  training? 
Other  topics  include  how  to  portray  battlefield  synchronization,  how  to  determine  feedback 
effectiveness,  and  the  identification  of  training  challenges. 

(2)  Tools  for  multi-Service  training  feedback  or  After- Action  Review  (AAR).  What 
different  measurements  and  types  of  feedback  are  useful  for  assessing  collective  performance? 
What  combinations  of  tools  and  methods  for  viewing  and  interpreting  events  (e.g.,  "bird's-eye 
views,"  combat  process  and  outcome  measures)  are  most  meaningful? 

(3)  A  training  program  using  existing  DIS  technologies.  The  MDT2-DIS  environment 
needs  an  open  system  architecture  and  accepted  conventions  for  interoperability  to  accommodate 
changes  needed  to  improve  training.  Costs  and  effectiveness  must  be  estimated  to  determine  the 
value  of  such  training. 

Overall,  MDT2  addresses  the  need  to  integrate  training  methods  and  tools  with  successful 
engineering  of  a  DIS  network  in  order  to  provide  significant  training  utility. 

1.1.2  Purposes  of  MDT2-CAS 

The  MDT2-CAS  project  was  the  first  one  performed  in  a  multi-Service  testbed  that 
sought  to  determine  what  training  and  feedback  works  well,  what  does  not,  and  what  needs  to  be 
improved  to  take  best  advantage  of  the  technology.  It  investigated  a  schema  for  best  use  of  DIS 
technology  to  provide  practical  and  cost-effective  home  station  training  in  addition  to  field 
exercises.  Except  for  a  few  feasibility  demonstrations,  the  application  of  DIS  technology  to 
training  across  the  military  Services  is  new. 

Specifically,  the  project  objectives  for  MDT2-CAS  were  (a)  to  develop  recommendations 
for  preparing  training  objectives  and  designing  performance  measures  for  multi-Service  CAS, 
and  (b)  to  demonstrate  the  value  of  using  DIS  technology  for  CAS  training.  "Lessons  learned" 
from  this  experience  provide  the  basis  for  these  recommendations  and  their  use  for  other  types  of 
training  in  multi-Service  tactical  operations. 

During  its  first  2  years,  MDT2  showed  the  feasibility  of  using  DIS  technology  to  train 
multi-Service  CAS  combat  operations.  CAS  operations  were  trained  and  that  training  was 
documented  with  measures  of  performance.  The  project  accomplishments  included  development 
of  a  testbed  including  (a)  training  objectives  and  scenarios  for  CAS,  (b)  a  distributed  network  to 
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conduct  such  training,  and  (c)  tools  and  methods  to  measure  performance  and  provide  training 
feedback. 

1.2  Background 

1.2.1  Principles  of  Distributed  Interactive  Simulation  (PIS) 

No  single  simulation  site  had  the  numbers  and  types  of  simulators  needed  to  support  the 
multi-Service  training  objectives  of  MDT2.  Therefore,  simulators  at  different  sites  across  the 
United  States  were  linked  with  one  another  to  provide  the  real-time  interactive  capability  needed 
for  MDT2.  This  allowed  ground  and  air  participants  from  different  sites  to  interact  with  one 
another  for  planning  and  executing  simulated  combat  missions  as  part  of  a  warfighting  team. 

DIS  technology  provided  the  means  for  linking  the  different  simulators  used  in  MDT2. 
DIS  allows  real-time  interaction  between  simulators  regardless  of  whether  these  simulators  are  in 
adjacent  rooms  or  thousands  of  miles  apart. 

A  DIS -based  exercise  relies  on  a  network  of  simulations  and  simulators  (manned, 
computer-generated,  or  both)  as  well  as  supporting  computer  and  communication  technologies. 

It  represents  an  extension  of  the  Simulator  Networking  (SIMNET)  program  developed  by  the 
Defense  Advanced  Research  Projects  Agency DIS  relies  on  the  following  principles  greatly 
simplified  from  more  detailed  documents:^ 

(1)  There  is  no  central  computer  that  tells  each  simulator  how  its  actions  influence  others. 

(2)  Each  simulator  broadcasts  its  state  (e.g.,  dead  or  alive),  and  changes  in  its  location, 
movement,  and  action  to  other  simulators  using  standardized  protocols.  Receiving  simulators 
detect  broadcast  information  and  calculate  how  it  affects  them. 

(3)  Each  simulator  establishes  or  dead-reckons  the  position  of  other  simulators.  Each 
simulator  maintains  a  dead-reckoning  model  of  other  simulators  and  regularly  compares  its  actual 
state/position  with  that  model.  Whenever  a  significant  difference  exists,  the  simulator  broadcasts 
update  information. 

(4)  Each  simulator  creates  an  appropriate  virtual  world  based  on  the  information  received 
from  other  simulators  and  its  own  stored  information.  Each  simulator  depicts  its  virtual  world, 
using  its  unique  simulator  technologies  (e.g.,  computer  image  generation,  display, 
communication,  and  computers). 


“  Alluisi,  E.  A.  (1991).  The  development  of  technology  for  collective  training:  SIMNEH',  a  case 
history.  Human  Factors,  33,  343-362. 

^  Institute  for  Simulation  and  Training,  University  of  Central  Florida  (1994).  Standard  for 
Distributed  Interactive  Simulation  -  Application  Protocols,  Version  2.0  Fourth  Draft  (Revised), 
Orlando,  FL:  Author. 


3 


1.2.2  Overview  of  the  MDT2-CAS  Pro2ram 


Connecting  the  simulators  of  various  Armed  Services  over  a  distributed  interactive 
network  allows  personnel  at  widely  diverse  locations  to  train  together  on  certain  tactics, 
techniques,  and  procedures  (TTP)  without  leaving  home  station.  Training  the  TTP  for  CAS  was 
chosen  for  the  initial  MDT2  training  exercises  because  CAS  can  involve  all  four  Services,  and 
each  Service  has  simulators  that  replicate  combat  environments  for  CAS.  In  addition,  CAS 
operations  are  extremely  difficult  to  train  in  any  environment  other  than  DIS. 

There  were  two  sets  of  MDT2-CAS  training  exercises,  one  in  May  1994  and  one  in 
February  1995,  which  linked  simulations  and  simulators  of  three  Services  over  the  Defense 
Simulation  Internet  (DSINet)  and  one  additional  T-1  (56Kb)  telephone  line.  Air  Force  pilots  at 
Armstrong  Laboratory,  Mesa,  AZ  (formerly  Williams  Air  Force  Base),  and  Marine  Corps  pilots 
at  the  Air  Combat  Environment  Test  and  Engineering  Facilities  of  the  Naval  Air  Warfare  Center 
Aircraft  Division  (NAWCAD),  Patuxent  River,  MD,  flew  simulators  that  emulated  attack  and 
forward  air  control  aircraft.  They  supported  the  conduct  of  ground  force  operations  by  Army 
units  and  Air  Force  liaison  elements  in  vehicle  simulators  and  a  Tactical  Operations  Center 
(TOC)  at  the  Mounted  Warfare  Test  Bed  (MWTB),  Fort  Knox,  KY.  In  addition,  a  Marine 
ground  laser  designator  team  served  as  a  ground  Forward  Air  Controller  (FAC)  and  target 
designator  using  the  Deployed  Forward  Observer/  Modular  Universal  Laser  Equipment 
(DFO/MULE)  simulator  located  in  May  1994  at  San  Diego,  CA  and  in  February  1995  at  Mesa, 
AZ.  VIPs  observed  the  exercise  at  the  sites  or  at  a  special  monitoring  facility  at  the  Institute  for 
Defense  Analyses  (IDA),  Alexandria,  VA.  The  total  number  of  trainees  from  the  Services  in 
each  of  the  MDT2-CAS  exercises  was  19.  (There  were  also  14  drivers  and  gunners  for  the 
Abrams  tank  and  Bradley  infantry  vehicle  simulators  at  Fort  Knox,  who  participated  in  the 
exercises  but  whose  performance  was  not  crucial  for  training  CAS.)  Figure  1  shows  the  MDT2- 
CAS  Training  Exercise  configuration  for  Eebruary  1995. 


IDA 


Figure  1.  Multi-Service  Distributed  Training  Testbed  for  Close  Air  Support. 
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The  training  exercise  wzis  conducted  over  a  5 -day  period  including  multiple  battles  and 
feedback  about  performance.  On  the  first  day,  all  trainees  were  briefed,  received  familiarization 
training  in  their  simulators,  and  participated  in  an  offensive  battle  against  semiautomated  forces 
(SAPOR)  controlled  by  subject  matter  experts  (SMEs).  Over  the  succeeding  4  days,  two 
defensive  battles  and  two  offensive  battles,  one  per  day,  were  conducted. 

During  each  battle,  data  about  battle  procedures  and  outcomes  were  collected  to  support  a 
multi-Service  After  Action  Review  (AAR)  to  give  trainees  feedback  on  their  performance.  Data 
were  obtained  in  two  ways:  (1)  CAS  experts  used  checklists  and  rating  forms  to  assess  combat 
procedures,  and  (2)  electronic  systems  recorded  performance  and  outcomes.  These  performance 
assessment  methodologies  and  resulting  recommendations  are  presented  later  in  greater  detail 
(Section  2.4.2).  After  the  battle,  an  AAR  was  conducted. 

Each  evening  and  the  following  morning,  trainees  developed  plans  and  operations  orders 
for  the  upcoming  battle.  The  battle  was  then  conducted,  an  AAR  was  held,  and  preparations 
were  made  for  the  next  battle.  Performance  improved  with  each  battle  in  each  of  the  key  training 
objectives  for  the  exercise. 

1.3  Presentation  of  the  Recommendations 


The  following  four  sections  recommend  practices  for  planning  and  conducting  multi- 
Service  training  and  for  developing  a  prototype  training  program  using  DIS  technology.  The 
recommendations  are  drawn  from  experience  gained  in  MDT2-CAS  combined  with  extensive 
prior  R&D  about  training.  The  final  section  outlines  a  full  set  of  steps  that  can  be  used  for 
planning  and  coordinating  this  kind  of  exercise. 


2.0  DESIGNING  AND  PLANNING  THE  TRAINING  EXERCISE 
2.1  Development  of  a  Project  Baseline 

Recommendation;  A  project  plan  integrating  training  requirements  with 

engineering  should  be  developed  for  all  participants  to  use  as  a  baseline. 

The  program  director  produces  a  plan  that  includes  both  training  and  engineering 
requirements  following  recommendations  in  subsequent  sections  of  this  report.  This  plan  should 
cover  exercise  objectives,  mission  scenarios,  and  functional  requirements.  It  becomes  the 
reference  document  for  all  subsequent  development  and  implementation.  Undoubtedly,  this  plan 
will  change  as  the  training  objectives  and  data  requirements  mature  and  as  simulation 
components  are  modified.  These  changes  need  to  be  understood  by  the  trainers,  SMEs,  and 
engineers,  with  plan  revisions  published  as  necessary  to  keep  everyone  informed. 
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2.2  When  to  Use  a  PIS  Environment  for  Tactical  Training 


The  reasons  to  use  DIS  technology  for  training  are  closely  related  to  those  for  individual 
simulators.  The  recommendations  that  follow  were  derived  from  surveys  and  interviews  with 
MDT2-CAS  participants,  requirements  for  training  CAS,  and  observations  by  the  authors  of  this 
report.  The  first  recommendation  deals  with  the  purpose  and  training  features  of  a  DIS 
environment;  the  second  suggests  when  not  to  use  DIS  for  training. 

Recommendation:  The  use  of  a  DIS  environment  for  training  crews,  groups,  teams, 

and  units  should  be  considered: 

Purpose 

•  To  train  coordination  and  synchronization  such  as  planning  and 
executing  a  multi-Service  CAS  mission. 

•  To  prepare  for  other  training,  such  as  for  the  National  Training 
Center  (NTC)  field  exercises  or  for  Air  Warrior. 

To  do  part-mission  training,  such  as  coordinating  CAS  pilots  with 
forward  air  controllers. 

•  To  conduct  training  otherwise  restricted  by  security,  safety, 
environment,  or  weather. 

Training  Features 

•  When  the  operation  being  trained  would  benefit  from  weapons' 
effects  in  more  realistic  time  and  spatial  relationships  to  enemy  and 
friendly  forces. 

•  When  real-time  kill  removal  and  casualty  assessment  would  enhance 
learning. 

•  When  situation  awareness  training  is  desirable  for  how  and  when 
fratricide  occurs. 

•  When  the  greatly  improved  feedback  of  DIS  environments  compared 
to  others  is  desired  for  participants. 

•  When  high  training  efficiency  (a  high  rate  of  tasks/events  trained  in  a 
short  amount  of  time)  is  desired. 


Recommendation;  The  use  of  DIS  for  multi-Service  applications  should  not  be 
considered  to: 

•  Train  Service-specific  tasks. 
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*  Replace  field/command  post  exercises. 

2.3  Participants  and  Objectives  for  PIS  Training  Development 

2.3.1  Responsibilities 

Recommendation:  A  multidisciplinary  team  should  be  set  up  to  support  the 
formulation  and  development  of  a  DIS-based  training  program. 

This  team  should  include  training  developers,  human  performance  experts,  network 
engineers,  site  representatives,  SMEs,  observer/controllers  (0/Cs),  and  representatives  of  the 
training  audience.  Throughout  the  program  development  process,  specialists  must  regularly 
interact  to  ensure  that  development  of  the  entire  system  and  its  operation  is  according  to  plans. 
For  example,  early  program  decisions  should  be  based  on  input  from  training  developers  on 
training  requirements  and  from  engineers  on  technologies  available  to  support  training. 

Moreover,  DIS  technology  must  be  assessed  at  each  stage  of  development  because  there 
are  likely  to  be  constraints  on  what  can  be  accomplished  that  were  not  recognized  at  the  outset. 
Engineers  using  a  prototypical  training  scenario  can  determine  which  training  tasks  may  be 
supported  fully  and  reliably  by  the  available  simulators.  Engineers  and  other  site  representatives 
can  provide  input  about  site  capabilities  and  limitations  relative  to  the  tasks  to  be  trained,  and  on 
whether  intersite  communications  are  adequate  for  exercise  control,  assessment,  and  feedback. 

Recommendation:  Technical  specialists  should  be  identified  as  primary  points-of- 
contact  for  various  phases  of  exercise  planning  and  execution. 
Responsibilities  and  authority  should  be  deRned  and  reporting  procedures 
established. 

A  DIS  exercise  hinges  on  the  integrated  efforts  of  several  different  teams.  The  typical 
program  director  does  not  have  enough  time  and  technical  knowledge  to  successfully  manage 
these  teams.  Therefore,  technical  focal  points  are  needed  to  coordinate  related  sets  of  activities 
such  as  enhancing  simulators  or  developing  training  materials.  Primary  and  backup  contacts 
should  be  identified  at  each  site  to  coordinate  site  preparation  and  participation;  in  addition,  a 
single  point-of-contact  at  each  site  must  handle  network  scheduling,  system  testing,  and  exercise 
control.  Preparations  involve  many  different  government  organizations  and  their  supporting 
contractors.  Special  procedures  must  be  established  for  tasking  work  among  organizations  and 
for  reporting  progress  in  terms  of  schedules,  technical  performance,  and  budgets. 
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Each  organization  involved  in  the  MDT2-CAS  program  had  primary  responsibility  for 
selected  aspects  of  the  exercise  and  its  preparation.  The  Air  Force  had  primary  responsibility  for 
network  engineering:  system  integration,  network  scheduling,  and  system  testing.  The  Army 
had  responsibility  for  training  exercises:  scenario  development,  exercise  conduet,  and  post¬ 
exercise  review  of  the  training  system’s  quality.  The  Navy  had  responsibility  for  training 
assessment:  trainee  performance  measurement  and  analysis.  A  program  director  had  sole 
responsibility  for  monitoring  all  work  and  budgets. 

2.3.2  Goals.  Tasks  and  Training  Objectives 

Recommendation;  Exercise  goals  should  be  specified  including  who  will  be  trained 
for  what  purpose. 

Commanders  of  Army  and  Air  Force  elements  scheduled  for  eooperative  field  exercises 
may  decide  to  prepare  using  simulator  training  for  relevant  staff  and  pilots.  The  result  will  be 
better  coordination  of  CAS  planning  and  execution  to  integrate  with  other  combat  assets  in  the 
field. 


Recommendation;  Exercise  planners  should  ensure  that  multi-Service  elements  are 
proficient  in  individual  and  Service-specific  skills. 

Inter-Service  and  group  training  should  be  scheduled  after  trainees  have  gained  some 
level  of  proficiency  with  individual  skills.  Observations  during  MDT2-CAS  exercises 
supplemented  by  other  experience  with  training  crews  and  groups  indieate  that  such  training 
suffers  when  individual  skills  are  deficient.  MDT2-CAS  used  trainees  who  had  CAS  planning 
and  execution  skills  in  their  own  Services,  but  most  had  very  limited  experience  in  multi-Service 
operations  since  opportunities  for  that  kind  of  activity  are  rare. 

Recommendation;  Identify  tasks  that  focus  on  interactions  across  Services  early  in 
exercise  planning. 

A  primary  driver  for  developing  DIS  technology  is  that  it  provides  opportunities  to 
practice  "warfighting  skills;"  that  is,  skills  that  go  beyond  basic  tactics  and  use  of  equipment  in 
multi-Service  and  combined  force  operations.  Early  selection  is  essential  because  these  tasks  are 
the  foundation  for  developing  training  objectives,  exercise/scenario  content,  performance 
measures,  and  methods  for  feedback. 

Recommendation;  Training  objectives  should  be  selected  or  developed. 

Training  objectives  clarify  the  types  of  tasks  to  be  trained.  A  comprehensive  set  of 
objectives  listed  later  in  this  doeument  was  developed  for  CAS  multi-Service  training  at  the 
Army  battalion  level.  At  the  present  time,  there  are  no  approved,  detailed,  and  measurable 
training  objectives  available  for  multi-Service  missions. 
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Recommendation ;  Based  on  the  training  objectives,  a  comprehensive  task  analysis 
should  be  performed  and  the  tasks  where  two  or  more  Services  have  to 
coordinate  or  cooperate  should  be  listed. 

Starting  from  scratch  with  no  existing  training  objectives  is  somewhat  daunting.  The  way 
to  begin  is  by  using  "doctrinally  correct"  TTPs  as  a  basis  for  developing  training  objectives. 

Build  a  list  of  critical  events  and  tasks  that  would  be  expected  during  the  course  of  an  operation 
and  use  this  to  put  together  a  preliminary  set  of  training  objectives. 

To  determine  the  basic  list  of  critical  events  and  tasks,  it  is  essential  that  experienced 
SMEs  conduct  a  thorough  review  of  existing  doctrine  and  TTP  (Service,  multi-Service,  and 
Joint).  The  product  of  this  review  should  be  a  list  of  events  with  standards  of  effective 
performance  for  each  one.  For  MDT2-CAS,  the  review  produced  a  comprehensive  list  of  critical 
events/tasks  at  Service  interfaces,  in  question-and-answer  format.  The  questions  asked  who  the 
participants  were  and  what  must  occur  among  them  to  ensure  doctrinally  correct  operations. 

The  focus  on  CAS  led  to  the  development  of  25  training  objectives  (Figure  2).  These 
were  used  to  develop  relevant  scenarios  and  performance  measurement  tools.  They  drove  the 
AAR  content  and  format  that  identified  multi-Service  performance  strengths  and  weaknesses 
relative  to  each  training  objective.  Figure  3  shows  an  expanded  training  objective  which 
includes  players,  conditions,  actions,  and  outcomes.  The  complete  set  is  in  Appendix  A. 

Recommendation;  Materials  prepared  to  this  point  should  be  double  checked  for 
correctness  with  Service  doctrinal  specialists  and  commands.  The  final 
training  objectives  can  then  be  written. 

For  CAS,  the  relevant  review  agencies  were  the  Air-Land-Sea  Application  Center 
(ALSA),  Langley,  VA  and  the  Air-Ground  Operations  School  (AGOS),  Hurlburt  Field,  FL. 
Upon  receipt  of  their  comments,  the  final  list  of  25  training  objectives  was  refined-each  with  a 
description  of  the  objective,  the  key  players  involved  with  that  objective  at  that  interface,  the 
conditions  for  performing  the  objective,  the  correct  actions  required,  the  desired  behavioral 
outcome  or  product  resulting  from  those  actions,  and  the  doctrinal  references  involved.  The  next 
step  was  for  experts  to  prioritize  the  training  objectives  to  insure  effective  use  of  exercise  time 
and  that  AARs  focus  on  key  issues. 

The  steps  above  may  sound  rather  simple;  however,  it  takes  a  major  effort  to  determine 
all  the  pieces  of  any  multi-Service  operation  and  how  they  interact.  This  is  the  key  to 
determining  everything  else  that  is  necessary  for  training  (i.e.,  scenarios,  control  mechanisms, 
performance  measurement,  AARs,  etc.). 

2.4  Functional  Requirements 

2.4.1  Scenario  and  Mission  Design 
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Recommendations:  How  to  get  started  on  scenario  and  mission  desiyfn: 


Describe  the  strategic  setting  for  combat  operations  and  specify  the 
enemy  and  friendly  forces.  Work  from  the  training  objectives. 

Involve  commanders,  trainers,  O/Cs,  and  engineers  as  early  as 
possible.  This  reduces  the  need  to  revise  scenario  materials  and 
decisions  as  each  separate  group  reviews  them. 

Select  missions  or  mission  slices  and  unit  composition  needed  to 
satisfy  training  objectives.  Decide  which  unit  elements  should  use 
simulators  and  which  should  be  simulated  by  SAFOR. 
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Number 

Title 

1 

Determine  battalion  mission  intent  and  concept  of  operation 

2 

Determine  the  enemy  situation 

3 

Develop  CAS  target  priorities 

4 

Develop  priority  of  intelligence  collection  assets  to  detect  CAS  targets 

5 

Integrate  CAS  and  other  fire  support  elements  with  maneuver  actions 

6 

Institute  fire  support  control/coordination  measures 

7 

Initiate  airspace  coordination  areas  (ACAs) 

8 

Incorporate  SEAD  in  the  fire  plan 

9 

Protect  laser  team 

10 

Prepare  a  decision  synchronization  matrix 

11 

Establish  methods  to  identify  targets  during  CAS  operations 

12 

Establish  methods  to  identify  fi'iendly  troops  during  CAS  operations 

13 

Conduct  a  fire  support/CAS  rehearsal 

14 

Pass  preplanned  CAS  targets  to  higher  headquarters 

15 

Prioritize  all  CAS  requests  from  subordinate  commanders 

16 

Pass  immediate  targets  and  on-call  target  updates  to  higher  headquarters 

17 

Provide  initial  brief  to  pilots  and  controllers 

18 

Update  airborne  pilots  as  necessary 

19 

Perform  communications  check  among  all  fire  support  and  CAS  participants 

20 

Control  CAS  air  attack 

21 

Confirm  status  of  friendly  air  defense 

22 

Arrive  on  station  and  establish  initial  communications 

23 

Synchronize  CAS  attack  with  other  direct  and  indirect  fires 

24 

Conduct  CAS  attack 

25 

Return  from  and  assess  CAS  mission 

Figure  2.  Training  Objectives 
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Key  Players  (FSO  is  the  principal  one): 


•  Bn/TF  Fire  Support  Officer  (FSO)  •  BN//TF  Staff  Intelligence  Officer 

•  Air  Liaison  Officer  (ALO)  (S-2) 

•  Bn/TF  Staff  Operations  Officer  (S-3)  •  Subordinate  commanders 

•  Airborne  Forward  Air  Controller  (Air-FAC)  •  CAS  pilots 

Conditions:  Enemy  air  defense  assets  haye  been  identified  and  are  located  in  the  Bn/TF  area. 

(Normally,  such  assets  will  be  mobile.) 

Actions:  The  FSO  takes  the  following  actions  that  are  releyant  to  planning  for  the  suppression  of 

enemy  air  defense  (SEAD): 

•  FSO  coordinates  with  the  S-2  and  S-3  to  identify  air  defense  assets  and  probable 
locations. 

•  During  planning,  FSO  plots  probable  locations  for  suppressiye  indirect  fires.  FSO  also 
plans  for  subordinate  commanders  to  use  against  air  defense  targets  of  opportunity.  Indirect 
fires  must  conform  with  ACA. 

•  During  execution,  FSO  coordinates  SEAD  fires  with  the  CAS  deliyery,  such  that  fires 
impact  1  minute  before  the  strike  and  continue  for  1  minute  after  the  aircraft  haye  departed. 

•  FSO  synchronizes  timing  of  air  attack  by  relaying  information  about  SEAD  through  the 
ALO  to  CAS  pilots  and  the  Air-FAC. 

Behayioral  Outcome  or  Product:  Enemy  air  defense  capability  is  effectiyely  suppressed.  No 

aircraft  are  lost  to  enemy  air  defenses. 

Reference: 

•  ArmyFM6-20-40,  Fire  Support  for  Brigade  Operations  (Heavy). 


Figure  3.  Example  of  a  Training  Objectiye:  8.  Incorporate  SEAD  in  the  fire  plan. 
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•  Obtain  copies  of  actual  operations  and  tasking  orders  from  higher 
headquarters  and  adapt,  as  necessary.  (Use  of  such  orders  adds 
realism  and  provides  familiar  recommendations,  Standard  Operating 
Procedures,  and  instructions.) 

•  Check  each  preceding  step  against  training  objectives  to  ensure  a 
close  match.  For  example,  do  not  use  a  bigger  mission  slice  or  more 
players  than  necessary  to  meet  key  training  objectives. 

•  Factor  in  cross-Service  conditions.  For  example,  plan  concurrent 
training  missions  outside  the  main  battle  for  units  or  unit  eiements 
waiting  to  participate  in  the  main  battle. 

Preparation  of  the  MDT2-CAS  scenario  began  with  the  "story"  of  the  combat  operation 
and  the  delineation  of  the  forces  involved.  Within  that  context,  the  defensive  and  offensive 
combat  operations  were  described  for  an  armored  battalion  task  force  that  was  part  of  an  armor 
brigade  attached  to  a  Marine  Expeditionary  Force  (as  in  Desert  Storm).  The  battalion's  maneuver 
and  fires,  with  emphasis  on  CAS,  were  also  presented  in  story  form,  giving  the  actions  of  the 
players  as  correct  examples  of  what  could  occur. 

From  this  series  of  stories,  the  scenario  was  refined  by  writing  Joint  Task  Force,  Marine 
Expeditionary  Force,  and  Army  Division  concepts  of  operations  and  commanders'  guidance. 
Detailed  brigade  operation  orders  were  developed  to  confine  the  relative  "free  play"  of  the 
battalion  within  the  technical  capability  of  the  simulation  environment.  Initial  positions  for 
friendly  and  enemy  forces  were  selected  and  a  series  of  actions  to  insert  during  play  were 
constructed  to  ensure  that  the  desired  range  of  CAS  missions  for  key  training  objectives  would 
occur  (see  Appendix  B). 

2.4.2  Performance  Assessment 


Assessment  Tools 

Effective  training  requires  that  combat  performance  (process  measurement)  and  combat 
outcomes  (outcome  measurement)  be  assessed.  Process  measures  focus  on  what  happens 
moment-to-moment  during  training;  outcome  measures,  on  the  other  hand,  provide  indicators  of 
combat  results.  Such  an  assessment  is  needed  to  (a)  diagnose  performance  deficiencies,  (b) 
specify  training  fixes,  (c)  plot  the  learning  curve  across  exercises  for  teams  or  groups  of  teams 
constituting  larger  units,  and  (d)  provide  performance  feedback  to  training  participants  in  order  to 
improve  subsequent  performance. 

The  MDT2-CAS  program  developed  multi-Service  process  and  outcome  measurement 
tools  and  methods  to  support  the  requirements  stated  above  for  a  DIS  environment. 

For  process  measurement,  tools  were  developed  to  assess  the  communication  and 
coordination  required  in  cross-Service  planning  and  execution  of  CAS.  These  measures  enabled 
0/Cs  for  an  exercise  to  record  whether  what  should  have  happened,  did  happen  (see 
Appendix  C). 
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Recommendation;  For  process  measurement,  two  tools  are  recommended: 

(1)  Checklists  allow  subject-matter  experts  to  make  structured  observations 
during  performance  to  record  what  happened,  and  (2)  judgments  about 
communication  and  coordination  quality  allow  experts  to  record  how  well 
performance  was  carried  out  in  relation  to  mission  requirements. 

Recommendation;  The  process  measures  should  be  completed  during  training 
exercises  and  used  to  help  prepare  After-Action  Reviews. 

The  checklist  data  can  be  used  to  measure  overall  trends  for  a  whole  exercise  or  for  parts 
of  the  battle.  In  one  method,  the  proportion  of  behaviors  performed  correctly  is  calculated 
relative  to  the  total  set  of  behaviors  that  should  have  been  performed.  This  provides  a  global 
indication  of  performance.  In  the  second  method,  groups  of  behaviors  are  divided  into 
functionally  related  clusters-for  example,  those  pertinent  to  a  specific  training  objective.  These 
clusters  can  then  be  examined  separately  to  obtain  an  indication  of  how  teams  performed  within 
the  functional  areas. 

Judgments  by  experts  identify  strengths  and  weaknesses  and  rate  teamwork  on  such 
dimensions  as  Communication,  Coordination,  Adaptability,  and  Situation  Awareness.  For 
example,  in  MDT2-CAS,  Communication  was  divided  into  factors  such  as  format,  terminology, 
clarity,  and  acknowledgments.  For  each  factor,  the  O/Cs  provided  comments  highlighting 
critical  teaching  points  for  feedback.  To  track  performance  trends,  the  0/Cs  also  provided 
ratings  of  how  well  the  key  participants  interacted  with  each  other  on  each  dimension.  The 
ratings  ranged  from  "1"  (Needs  Work)  to  "4"  (Outstanding). 

Recommendation;  Objective  measures  are  needed  to  summarize  battle  outcomes. 
Such  measures  include  casualties  by  weapon  system,  loss  exchange  ratios, 
and  amount  of  terrain  controlled  (see  also  Appendix  D). 

Recommendation;  Preliminary  planning  for  DIS-based  training  should  include 
coordination  with  a  training  measurement  specialist.  Determine  the 
information  and  displays  needed  to  conduct  effective  AARs. 

For  measuring  combat  outcomes,  the  MDT2  program  modified  and  used  the  computer- 
based  Unit  Performance  Assessment  System  (UPAS)^.  This  system,  implemented  on  a  personal 
computer,  stores  and  displays  information  (from  the  DIS  network)  about  engagements,  casualties, 
and  movement  of  simulated  vehicles.  It  supports  animated  replays  of  movement  and  firing 
events,  data  summary  graphs  and  tables,  a  timeline  of  exercise  movement  and  firing  events,  a 
battle  flow  tracing  movement  of  individual  vehicle  simulators,  and  a  "firefight  display" 
summarizing  direct  and  indirect  fire  impact  and  firing  vector  data.  The  UPAS  integrates  network 
data  with  terrain  data  and  unit  planning  data  (e.g.,  control  measures)  to  provide  a  more 
meaningful  description  of  events  than  can  be  provided  by  network  data  alone. 


®  Meliza,  L.L.,  Bessemer,  D.W.,  and  Tan,  S.C.  (1994).  Unit  Performance  Assessment  System 
(UPAS)  Development.  US  Army  Research  Institute  Technical  Report  1 008. 

\  \ 
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For  MDT2-CAS,  outcome  measures  for  missions  were  stated  in  terms  of  the  number  of 
enemy  vehicles  damaged  or  destroyed,  the  number  of  bombs  released,  and  the  number  of  releases 
resulting  in  a  miss.  These  data  were  developed  for  the  entire  exercise  and  for  each  CAS  mission 
within  the  exercise.  In  addition,  two  types  of  data  displays  were  used  to  provide  information  on 
synchronization  of  fires.  The  first  was  a  graph  showing  volume  of  fires  by  ammunition  type,  and 
the  second  was  a  map  display  showing  the  locations  of  direct  and  indirect  fires.  Synchronization 
was  assessed  and  illustrated  by  the  volume  of  fires  by  weapon  system  over  time. 

Outcome  measures  need  to  be  used  with  caution  to  prevent  inaccurate  conclusions  about 
training  success.  In  the  MDT2-CAS  exercises,  the  accuracy  of  outcome  measures  depended  on 
how  they  were  registered  and  stored  by  UPAS  and  interpreted  by  0/Cs.  Near  misses  or  damage 
to  a  target  was  particularly  difficult  to  record.  Casualty  data  can  help  to  trigger  discussions  about 
performance  problems  or  to  highlight  battle  trends.  The  focus  should  not  be  on  the  casualty  data 
itself  in  current  DIS  environments. 

Management  of  Performance  Assessment 

Recommendation;  Ensure  that  monitoring  stations  will  permit  O/Cs  to  control 
exercises  and  observe  performance. 

In  DIS  training  environments,  one  or  more  0/Cs  may  be  needed  to  evaluate  performance 
at  each  of  multiple  nodes.  Information  must  be  provided  to  0/Cs  to  enable  them  to  evaluate 
performance  at  both  the  site-specific  (local)  and  DIS  (global)  level.  The  local  view  gives  a 
different  battle  viewpoint  based  on  each  site.  Each  site  or  one  central  site  ideally  should  have 
access  to  all  views  to  develop  a  global  perspective  of  the  battlefield.  This  global  perspective  was 
not  available  for  MDT2-CAS.  However,  information  sharing  among  sites  should  be  designed  to 
provide  sufficient  and  equivalent  capabilities  for  0/Cs  to  meet  observation  requirements. 

As  systems  are  developed,  information  requirements  for  each  0/C  position  should  be 
analyzed.  Candidate  technologies  that  should  be  in  place  include:  (a)  a  communication  "hotline" 
between  O/Cs  during  an  exercise  to  ensure  that  problems  are  identified  in  timely  fashion  and  that 
control  of  the  scenario  is  maintained,  (b)  displays  for  ready  and  rapid  monitoring  of  node  status 
in  DIS,  and  (c)  technologies  that  enhance  observation  of  multi-Service  interactions  such  as 
displays  which  can  switch  between  air  and  ground  perspectives  of  combat. 

For  scenario  control  and  observation  in  MDT2-CAS,  O/Cs  relied  on  displays  and  other 
interfaces  developed  for  simulator  systems  designed  originally  for  single-Service  use  and  fewer 
O/Cs.  O/Cs  at  the  Fort  Knox  site,  for  example,  competed  for  the  stealth  displays  for  perspective 
views  and  plan  view  situation  displays  They  were  needed  to  control  the  exercise  and  to  measure 
performance. 

Recommendation:  To  ensure  that  intended  measurements  or  observations  are 

obtained,  the  O/Cs’  workload  should  be  analyzed  to  determine  whether  they 
can  handle  a  dual  role  of  exercise  control  and  performance  assessment. 
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Individuals  responsible  for  observation,  control,  and  measurement  functions  in  an 
exercise  must  ensure  first  and  foremost  that  the  scenario  flow  unfolds  realistically.  When  their 
focus  must  shift  to  scenario  control,  for  example,  assessment  opportunities  will  be  lost.  An 
assessment  team  should  be  dedicated  exclusively  to  that  task  particularly  in  a  distributed  training 
system  where  scenarios  unfold  in  a  free-play  fashion,  or  where  system  reliability  can  be  suspect. 

In  MDT2-CAS,  opportunities  to  measure  performance  were  lost  because  of  the  O/Cs' 
high  workload.  For  example,  0/Cs  were  responsible  both  for  assessing  training  performance  and 
for  controlling  the  flow  of  a  battle  using  “trigger  events”  that  focused  trainees  on  selected 
objectives.  It  was  difficult  to  inject  trigger  events  such  as  an  immediate  CAS  mission  or  modi¬ 
fication  of  an  existing  Airspace  Coordination  Area  (ACA)  while  assessing  training  performance. 

2.4.3  Performance  Feedback  (Multi-Service  After-Action  Reviews) 


The  AAR  provides  direct  feedback  to  trainees  by  personally  involving  them  in  the 
assessment  process.  AARs  enable  the  trainees  to  discover  what  happened  during  the  exercise  as 
well  as  how  and  why  it  happened.  They  interact  with  one  another  and  with  0/Cs  at  all  the 
various  sites.  Figure  4  shows  the  0/C  organization  used  in  MDT2-CAS. 
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Figure  4.  Observer/Controller  (0/C)  Organization. 
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Rernmniendation;  One  person,  normally  the  senior  trainer  or  a  senior  O/C,  should 
be  in  charge  of  the  After-Action  Review  process.  That  person  should 
coordinate  the  process  from  initial  planning  through  development  and 
delivery. 

During  the  May  1994  MDT2-CAS  exercises,  an  extremely  capable  senior  trainer 
orchestrated  and  conducted  the  AARs.  However,  he  was  able  to  prepare  for  the  exercise  only  1 
week  in  advance  and  had  some  difficulty  maintaining  focus  on  the  "unusual"  multi-Service 
nature  of  the  AARs.  In  the  February  1995  MDT2-CAS  exercises,  the  senior  trainer  had 
developed  the  procedures  for  that  AAR,  and  had  played  a  major  role  in  developing  the  exercise 
training  objectives  and  the  training  scenario.  As  a  result,  the  AARs  were  more  focused  on 
training  objectives. 

Recommendation;  Planning  for  After- Action  Reviews  should  be  integrated  into  the 
exercise  scenario  at  logical  break-points,  and  the  amount  of  time  needed  for 
AAR  preparation  should  be  considered  in  setting  the  schedule. 

Recommendation;  Rehearsal  of  AARs  is  essential  to  enable  O/Cs  to  work  out 
training  points  and  supporting  documentation. 

For  MDT2-CAS,  the  logical  break-point  for  AARs  was  at  the  conclusion  of  each  battle. 
Each  AAR  required  about  2  hours  of  preparation  to  (a)  collect  feedback  comments  from  the  O/Cs 
at  each  site,  (b)  consolidate  these  comments  into  major  teaching  points,  (c)  work  out  the  means  to 
best  display  those  points,  using  audio  and  video-teleconference,  and  (d)  conduct  a  quick  rehearsal 
of  the  AAR.  The  AAR,  therefore,  was  conducted  2  hours  after  each  battle. 

During  rehearsals,  the  O/Cs  should  consolidate  comments  as  they  would  for  an  actual 
exercise  and  orchestrate  which  teaching  points  to  draw  out  of  the  trainees,  and  how  and  in  what 
order  to  best  display  those  points.  Rehearsals  before  an  exercise  can  be  accomplished  by 
replaying  data  logger  tapes  from  a  previous  exercise  or  by  having  SAFOR  operators  run  through 
the  exercise  scenario  while  controlling  not  only  enemy  forces  but  also  the  elements  to  be  played 
by  the  trainees  during  the  actual  exercise.  Site  O/Cs  should  look  for  key  teaching  points  to  be 
made  and,  after  the  mock  battle,  should  provide  comments  to  the  senior  trainer.  A  final  rehearsal 
should  be  held  with  the  following  personnel  present:  the  exercise  director,  senior  trainer,  site 
O/Cs,  SAFOR  operators,  simulator  technicians,  DSENet  operators,  and  video-teleconference 
(VTC)  equipment  operators.  It  is  during  this  rehearsal  that  both  technical  and  training-related 
problems  can  be  discovered  and  corrected,  and  presentation  methods  can  be  refined. 

Recommendation;  All  sites  should  use  the  lowest  common  communication 

technologies  to  support  distributed  AARs.  Planning  should  take  any  limiting 
factors  into  account. 

In  the  near  term,  technology  is  likely  to  be  a  factor  limiting  the  feedback  about 
multi-Service  performance  that  can  be  provided  to  multiple  sites.  For  MDT2-CAS,  these 
limitations  included  the  lack  of  common  plan  view  displays  as  well  as  other  performance 
displays,  and  the  inability  to  retransmit  recorded  voice  communications  for  the  purpose  of  the 
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AAR.  Therefore,  work-arounds  were  implemented  in  the  exercise.  The  plan  view  and  other 
performance  and  briefing  displays  were  transmitted  from  the  Fort  Knox  site  to  the  other  sites  via 
a  VTC  system  supplemented  with  a  conference  telephone  call  for  continuous  voice 
communications. 

Recommendation:  After-Action  Review  planning  should  include  identifying  the 

specific  locations  at  each  site  where  trainees  will  participate  in  AARs,  as  well 
as  the  requirements  for  equipment  and  supplies. 

Needed  items  such  as  speaker-telephones  for  teleconferencing  among  site  0/Cs  and  VTC 
equipment  for  conduct  of  the  AAR  ought  to  be  identified  early,  and  the  necessary  networks  to 
support  those  systems  arranged.  Video-teleconferencing,  in  particular,  is  an  evolving  technology 
with  incomplete  standards.  Even  within  the  same  product  line,  different  units  may  be 
incompatible  with  one  another~as  discovered  for  MDT2-CAS. 

Recommendation:  Methods  for  displaying  After-Action  Review  information  should 
he  planned  in  detail. 

DIS  technology  enables  a  variety  of  options  for  effective  displays.  For  MDT2-CAS,  the 
following  five  methods  were  used:  (a)  The  Unit  Performance  Assessment  System  (UPAS), 
discussed  earlier,  provided  overall  outcome  information  in  the  form  of  graphs,  charts,  and  map 
displays,  (b)  Harvard  Graphics—  charts^  (or  other  graphics)  were  used  to  display  teaching  points 
and  data,  (c)  The  Plan  View  Display  (PVD),  which  shows  a  two-dimensional  situation  map 
including  weapons'  effects  over  time,  showed  replays  of  portions  of  the  battle  at  normal  speed,  3- 
times  speed,  and  6-times  speed,  (d)  The  Stealth  View,  which  displays  a  three-dimensional 
perspective  of  the  battlefield  from  any  viewpoint,  was  replayed  to  reinforce  the  information 
provided  on  the  PVD.*  (e)  Digital  voice  replay  which  allows  trainees  to  hear  communications  of 
various  elements  at  key  points  during  the  battle.  This  voice  replay  was  run  concurrently  with  a 
PVD  or  Stealth  View  replay  for  added  emphasis. 

Which  display  method  to  use  and  when  to  use  it  should  be  determined  during 
preplanning.  Selection  should  be  based  on  anticipated  teaching  points  for  key  training 
objectives.  The  methods  used  should  highlight  those  points  in  a  clear,  understandable  manner 
that  elicits  discussion  from  the  trainees. 


’  The  use  of  brand  names  or  trade  marks  is  for  accurate  documentation  and  does  not 
constitute  endorsement  by  the  Department  of  Defense  or  its  Militaiy  Departments. 

®  Airspace  for  combat  flights  extended  beyond  the  limits  of  the  PVD  and  Stealth  display  used 
by  the  O/Cs  with  ground  forces  at  Fort  Knox.  Feedback  therefore  was  limited  to  this  smaller 
air  space  with  integrated  views  of  air  and  ground  operations. 
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Recommendation:  At  the  conclusion  of  each  After-Action  Review,  senior  trainers 
and  site  0/Cs  should  conduct  a  post-AAR  critique  (an  AAR  of  the  AAR). 

The  focus  should  be  on  what  went  well,  what  did  not,  and  what  to  change  or 
refine  for  the  next  AAR.  Comments  from  any  participant  should  be  solicited. 

Multi-Service  distributed  AARs  are  not  yet  well  developed  and  institutionalized  so  that 
critiques  are  essential  for  improvement. 

2.4.4  Component  or  Service-Specific  Debriefs 

Recommendation:  In  addition  to  the  multi-Service  AAR,  provisions  should  be  made 
for  component  or  Service-specific  debriefs.  These  debriefs  allow  the  detailed 
review  of  TTPs  that  are  not  of  interest  to  everyone  but  are  critical  to  specific 
team  and  crew  performance. 

Inter-Service  coordination  and  communication  skills  must  be  based  on  a  solid  foundation 
of  individual  and  Service-specific  skills.  Debriefs  about  these  latter  skills  ensure  that  individuals 
can  gain  the  most  benefit  when  they  train  with  other  Services.  Local  sites  should  use  the  best  and 
most  appropriate  technologies  and  displays  available  since  compatibility  with  others  is 
unnecessary.  An  added  benefit  of  such  debriefs  results  if  they  are  scheduled  for  periods  that 
otherwise  would  be  time  spent  waiting  for  the  exercise  to  continue. 

Recommendation:  To  ensure  maximum  benefit  from  the  use  of  simulator  resources, 
concurrent  training  should  be  planned  for  participants  when  they  are  not 
involved  in  the  DIS  exercise. 

When  not  tasked  for  MDT2-CAS,  the  F-16  pilots  and  DFO/MULE  team  used  their 
simulator  systems  for  other  training  tasks  such  as  precision-guided  munitions  delivery  under  the 
direction  of  a  site  0/C  who  debriefed  them  on  how  they  performed.  Likewise,  the  ground  force 
was  debriefed  on  any  fires  and  maneuvers  conducted  separately  from  CAS. 


3.0  TRAINING  EXERCISE  PREPARATION  AND  EXECUTION 
3.1  Training  Environment  and  Force  Structure 

Recommendations: 

•  Ensure  sufficient  correlation  of  terrain  databases  for  all  simulators 
and  simulations.  Arrange  engineering  fixes  to  translate 
inconsistencies  in  various  separate  simulator  databases. 

•  Identify  simulator  limitations  and  adjust  scenarios  or  exercise 
management  as  "workarounds."  Key  limitations  include  the  number 
of  vehicles  that  can  be  simulated  and  the  visibility  range. 
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•  Plan  alternatives  for  use  if  network  connections  are  interrupted.  For 

example,  designate  O/C  personnel  to  play  roles  at  sites  that  "fall  off 
the  net"  in  order  to  continue  training  at  remaining  sites. 

For  the  MDT2-CAS  terrain  database,  the  National  Training  Center  (NTC)  at  Fort  Irwin, 
CA  was  chosen  because  it  could  be  used  by  simulators  at  all  sites.  It  also  allows  for  future 
comparison  with  actual  NTC  field  exercises,  and  for  possible  use  in  combining  MDT2  exercises 
with  live  force-on-force  exercises. 

The  technical  restrictions  associated  with  various  simulators  had  considerable  effect  on 
the  planning.  For  example,  the  simulators  in  MDT2-CAS  could  not  all  display  the  same  number 
of  entities  such  as  enemy  and  friendly  vehicles  and  weapons  effects  in  the  same  battle  space. 
Scenarios  and  missions  had  to  be  constructed  so  that  the  number  of  entities  coming  into  the  fields 
of  view  of  those  simulators  did  not  exceed  their  limitations  (also  see  the  following  Section). 

Simulator  connectivity  among  the  networked  sites  improved  greatly  from  the  beginning 
to  the  end  of  the  MDT2-CAS  program.  O/C  personnel  at  Fort  Knox  used  voice  communications, 
stealth  displays,  and  SAPOR  to  emulate  other  sites  when  one  or  more  were  disconnected.  These 
workarounds  were  acceptable  although  the  net  effect  was  reduced  multi-Service  training. 

3.2  Simulation  and  Simulator  Environment 


MDT2  has  the  potential  for  training  multi-Service  tasks  involving  planning  and  execution 
at  the  tactical  level.  It  clearly  demonstrates  the  significant  progress  made  by  the  DIS  community 
in  developing  standards  for  compliance,  compatibility,  and  interoperability  of  simulators. 
However,  simulation  still  cannot  provide  unconstrained  training  of  all  mission-related  tasks. 
These  technology  limitations  must  be  considered  when  defining  the  possible  range  of  training 
objectives  and  mission  scenarios.  Appendix  E  contains  a  description  of  simulators  used  in 
MDT2-CAS. 

In  the  typical  DIS  exercise,  a  number  of  existing  simulators  are  linked.  Because  these 
simulators  were  built  to  meet  different  requirements,  they  differ  in  architectures,  displays,  image 
generators,  databases,  and  processing  capabilities.  In  addition,  DIS  protocols  are  not  yet  able  to 
transmit  the  full  complexity  of  the  combat  environment  between  simulators.  Consequently,  the 
virtual  worlds  produced  within  different  simulators  in  a  DIS  exercise  may  not  be  identical  for 
historical  and  technological  reasons  rather  than  for  operational  reasons. 

3.2.1  Interoperability  of  Simulators 

The  major  technical  challenge  in  implementing  an  exercise  with  current  DIS  technology 
is  to  ensure  that  the  trainees  using  different  weapon  system  simulators  can  interact  with  one 
another  as  part  of  the  same  battle.  Interoperability  is  a  fundamental  problem  in  designing  and 
implementing  a  DIS  exercise.  It  is  encountered  whenever  different  simulators  must  interact  with 
one  another  as  members  of  the  same  virtual  environment.  Their  representations  of  that 
environment  must  be  sufficiently  correlated  with  one  another  to  provide  the  appropriate  stimuli, 
responses,  and  feedback  necessary  for  training. 
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Recommendation;  The  capability  of  each  simulator  and  the  degree  of 
interoperability  among  them  should  be  determined. 

Trainers,  engineers,  and  SMEs  must  work  together  to  determine  technical  capabilities 
(and  limitations)  for  interoperability.  Engineers  must  define  simulator  capabilities  for  the 
conditions  under  which  the  simulators  will  be  used  and  the  degree  of  interactivity  that  is  expected 
among  the  various  sites.  Similarly,  trainers  can  fully  define  the  training  objectives,  mission 
scenarios,  and  data  requirements  only  on  the  basis  of  the  capabilities  of  the  specific  simulators 
and  support  systems.  To  facilitate  this  blending  process,  trainers  need  to  provide  the  engineers 
with  a  broad  overview  of  the  training  objectives,  the  mission  that  will  be  simulated,  a 
prototypical  scenario,  and  data  requirements.  This  information  enables  the  engineers  to 
understand  what  the  simulation  must  do.  They  then  can  describe  to  the  trainers  and  SMEs  the 
capabilities  and  limitations  of  each  simulator  from  the  perspective  of  the  particular  exercise. 

For  MDT2-CAS,  the  principal  source  of  interoperability  problems  involved  the 
appearance  of  the  battlefield  in  the  different  simulators.  In  the  real  world,  the  appearance  of  the 
battlefield  depends  primarily  on  a  weapons  system's  position  and  sensors.  For  example,  soldiers 
in  a  tank  cannot  see  as  far  as  pilots  flying  10,000  feet  above  ground-level.  Although  theoretically 
both  the  soldiers  and  the  pilots  have  the  same  information  available  to  them,  their  ability  to 
detect  and  use  this  information  differs  because  of  their  location  on  the  battlefield.  In  contrast, 
different  simulators  in  MDT2-CAS  introduced  artifacts  into  the  battlefield's  appearance  because 
of  differences  in  the  fidelity  of  terrain  representation,  and  the  characteristics  of  the  various 
simulators.  Consequently,  in  MDT2-CAS,  the  appearance  of  the  battlefield  depended  not  only 
on  the  weapon  system  being  simulated  and  its  simulated  location  on  the  battlefield,  but  also  on 
the  simulator  itself. 

Recommendation;  Technical  fixes  should  be  used  to  achieve  sufficiently 
correlated  terrain  databases. 

Given  the  current  state-of-the-art  in  simulation,  the  appearance  of  the  same  terrain  in 
different  simulators  is  likely  to  differ.  These  differences  must  be  identified  and  minimized  if 
different  simulators  are  to  share  a  realistic  common  virtual  world.  Correlation  tests  should  be  run 
from  selected  coordinates  during  preparation  for  an  exercise.  Elevation  and  intervisibility  data 
should  be  recorded  and  appropriate  database  or  scenario  modifications  initiated. 

Since  terrain  databases  are  developed  specifically  for  each  simulation  system,  there 
usually  will  be  differences  in  the  degree  of  terrain  resolution.  This  creates  a  lack  of  correlation 
between  terrain  databases  in  the  different  simulators.  A  result  of  this  limited  correlation  is  that 
ground  vehicles  can  be  positioned  above  ground  in  one  simulator  and  below  ground  in  another 
simulator.  In  MDT2,  this  problem  was  reduced  by  using  a  technical  fix  known  as  "ground 
clamping."  This  fix  allowed  the  receiving  simulator  to  position  the  entity  at  a  ground  elevation 
consistent  with  its  local  terrain  elevation  data.  It  also  kept  marking  rounds  and  bombs  from 
detonating  underground. 
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Although  ground  clamping  works  fairly  well  for  ground  vehicles,  it  does  not  solve  all  the 
problems  associated  with  mismatches  in  terrain  elevation.  They  may  still  cause  errors  in  line  of 
sight  and  may  cause  aircraft  to  "fly  through  hills"  when  viewed  from  ground  vehicles.  Therefore, 
in  addition  to  "ground  clamping,"  extensive  testing  needs  to  be  done  to  ensure  that  the 
performance  of  the  participants  and  their  weapon  systems  will  not  be  compromised. 

MDT2-CAS  used  terrain  databases  representing  the  NTC  with  different  implementations 
depending  on  a  site's  simulator  capabilities  and  requirements  for  the  exercise.  The  aircraft 
simulators  in  Mesa,  AZ  and  Patuxent  River,  MD  used  the  same  terrain  database;  the  DFO/MULE 
(laser  designator)  used  highly  detailed  elevation  data  overlaid  with  geo-specific  photographs 
taken  at  the  NTC.  The  MWTB  at  Fort  Knox  had  still  another  database  for  SIMNET  vehicles.  At 
times  during  the  course  of  an  exercise,  the  elevation  data  were  not  properly  matched.  Tests 
comparing  selected  coordinates  across  sites  were  done  during  exercise  preparations  to  achieve 
the  best  terrain  correlation  possible. 

Recommendation;  Technical  fixes  should  be  used  to  accommodate  differences 
among  simulators  in  displaying  moving  vehicles  and  weapons  effects. 

Differences  in  the  number  of  moving  vehicles  and  weapons  effects  that  can  be  displayed 
to  trainees  in  the  various  simulators  have  significant  impacts  on  the  design  of  scenarios  and  the 
conduct  of  training.  Scenarios  have  to  be  carefully  tailored  to  ensure  that  each  trainee  is  able  to 
respond  appropriately  as  the  battle  unfolds. 

MDT2-CAS  managed  simulator  differences  to  display  moving  vehicles  and  weapons 
effects  by  using  scenarios  with  limits  on  what  was  needed  for  realistic  combat.  While  the  armor 
simulators  at  the  MWTB  had  an  essentially  unlimited  ability  to  process  and  display  moving 
vehicles  and  weapon  effects,  all  the  other  simulators  had  much  more  limited  capabilities.  The 
aircraft  simulators  at  Mesa,  AZ  and  Patuxent  River,  MD  were  modified  to  be  capable  of 
processing  and  displaying  a  total  of  54  and  16  moving  vehicles  and  weapon  effects,  respectively. 
The  DFO/MULE,  however,  could  not  be  modified,  within  available  resources,  to  see  more  than 
five  moving  vehicles.  Scenario  action  was  designed  to  accommodate  the  simulators. 

Recommendation:  Compatible  representations  of  the  virtual  world  among  sites 
should  be  developed  by  testing  how  different  protocol  translators  and 
Network  Interface  Units  (NIUs)  handle  and  represent  DIS  Protocol  Data 
Units  (PDUs). 

Differences  in  how  the  various  sites  handle  DIS  PDUs  affect  both  the  simulation  and  the 
information  available  to  trainers  and  engineers.  During  MDT2-CAS,  scenarios  and  training  had 
to  accommodate  differences.  For  example,  the  site  configuration  at  Patuxent  River,  MD  for  the 
Air-FAC  did  not  process  fire  and  detonate  information  from  23mm  rounds  fired  by  the  ZSU-23/4 
at  the  MWTB.  The  Air-FAC,  therefore,  had  to  be  immune  from  engagement  by  that  ground 
threat.  Another  technical  difference  among  sites  was  that  the  DIS  laser  PDU  was  not  translated 
to  the  protocol  format  at  the  MWTB.  0/Cs  at  the  MWTB  could  not  determine  which  vehicles 
were  being  lased  and  were  not  able  to  use  that  tactical  information  for  trcdning  except  through 
verbal  reports  from  other  sites. 
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3.2.2  Exercise  Environment 


Rpcnmmendation;  The  training  objectives  and  mission  scenarios  should  be 
reviewed  in  terms  of  simulator  capabilities. 

Once  the  desired  exercise  requirements  and  technical  capabilities  have  been  defined,  it  is 
necessary  to  determine  whether  the  simulation  can  support  the  training  objectives  and  mission 
scenarios.  This  represents  a  continuation  of  the  interactive  exchange  of  information  among 
trainers,  engineers,  and  SMEs.  If  the  simulation  cannot  fully  support  the  training  objectives  and 
mission  scenarios,  decisions  must  be  made  regarding  the  importance  of  the  various  technical 
shortcomings  and  possible  solutions.  Solutions  range  from  modifying  the  training  objectives  or 
mission  scenarios  to  making  engineering  changes  to  one  or  more  of  the  simulation  systems 
within  time  and  budget  constraints. 

Recommendation;  A  communication  plan  should  be  developed  covering  all  types  of 
intersite  communications. 

Information  must  be  transmitted  between  DIS  sites  to  support  network  operations, 
exercise  control,  tactical  operations,  and  After-Action  Reviews.  The  need  to  provide  such  a  wide 
variety  of  information  imposes  a  number  of  communication  requirements  on  both  the  wide  area 
network  (WAN)  and  its  attached  local  area  networks  (LANs).  These  requirements  include 
identifying  what  communications  must  be  provided  between  which  sites  and  which  participants; 
estimating  bandwidth;  selecting  and  allocating  communications  media;  defining  and 
implementing  the  various  communication  links;  and  establishing  operating  procedures. 

Remmmendation:  For  geographically-distributed  exercises,  time  references  should 
be  established  and  all  participants  informed  what  they  are. 

Time  references  in  a  DIS  exercise  may  refer  to  simulated  time,  local  time  at  a  given  site 
(e.g.,  1000  MST),  local  time  at  another  site  (e.g.,  1200  EST),  or  a  standard  time  such  as  universal 
coordinated  time  (e.g.,  1700  GMT).  Confusion  is  particularly  likely  to  occur  when 
administrative  and  exercise  times  are  different. 

3.3  Exercise  Management 

3.3.1  Scheduling  and  Preparation 

Scheduling  a  DIS  training  exercise  is  similar  to  scheduling  a  field  training  exercise 
(FTX).  In  both  cases,  schedules  of  trainees  and  0/C  personnel  must  be  coordinated  and 
deconflicted.  For  an  FTX,  training  areas,  vehicle/aircraft/equipment  use,  and  supporting  fuel, 
munitions,  and  other  logistics  must  be  coordinated  and  deconflicted.  A  DIS  exercise  involves  a 
similar  set  of  scheduling  and  logistical  activities. 

Recommendation:  A  program  schedule  should  be  developed  after  the  training 
requirements  are  finalized. 
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Recommendation;  The  availability  of  simulations  and  simulators,  communication 
networks  (e.g.,  Defense  Simulation  Internet  or  DSINet),  and  supporting 
resources  must  be  coordinated  and  deconflicted  with  competing  demands. 

Trainee  participation  with  DIS  must  be  integrated  with  different  Services’  master  training 
schedules  as  far  ahead  of  the  scheduled  training  exercise  as  possible.  There  generally  is  no 
schedule  for  multi-Service  exercises  using  DIS  facilities,  so  planning  is  particularly  challenging 
for  major  units  such  as  battalions,  brigades,  squadrons,  and  wings. 

Recommendation;  Special  attention  needs  to  be  given  to  developing  an  exercise 
schedule  when  different  time  zones  are  involved. 

A  major  problem  in  scheduling  a  DIS  training  exercise  is  that  it  may  span  several  time 
zones.  Planners  must  consider  these  time  differences  in  developing  exercise  schedules.  Time 
differences  can  have  an  impact  on  all  participants.  For  example,  are  the  planned  times 
acceptable,  given  contractual  requirements  such  as  the  scheduled  periods  of  operation  and 
maintenance  for  simulation  and  networking  facilities?  It  is  very  easy  to  forget  that  an  exercise 
scheduled  to  kick  off  on  the  East  Coast  at  0900  EST  may  actually  require  personnel  on  the  West 
Coast  to  begin  working  at  0400  PST. 

3.3.2  Security  Requirements 

Recommendation;  Security  needs  must  be  considered  when  scheduling  DIS  training 
exercises. 

An  exercise  and  all  participating  sites  may  be  classified  even  if  only  one  site  on  the 
network  handles  secure  data.  This  may  require  upgrading  site  and  participant  security  clearances. 
Depending  on  the  clearance  level  required,  it  could  take  2-8  months  or  more  to  obtain  clearances 
for  all  the  participants.  The  MDT2-CAS  training  exercise  required  SECRET  NOFORN  for  each 
site  and  participant;  obtaining  the  necessary  clearances  took  approximately  2  months.  Had  it 
required  clearances  necessitating  a  complete  background  investigation,  6-8  months  would  have 
been  an  optimistic  forecast.  Check  with  network  security  officials  to  insure  necessary  lead  times 
when  making  initial  exercise  plans. 

3.3.3  System  Testing.  Rehearsals,  and  Participant  Prefraining 

Recommendation;  Functional  verifications  of  the  environment  should  be  conducted 
to  confirm  system  capabilities. 

Component  and  system  testing  must  be  conducted  as  necessary  to  verify  individual 
simulator  performance,  compliance  with  DIS  standards,  simulator  interoperability,  and 
compatibility  with  mission  scenarios.  In  addition,  stress  testing  should  be  performed  to  verify 
system  performance  under  exercise  loads.  Sufficient  time  must  be  scheduled  between  tests  to 
allow  engineering  and  scenario  modifications  to  be  made. 
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For  MDT2-CAS,  the  test  program  used  a  building  block  approach.  DIS  compliance  and 
simulator  interoperability  were  established  for  small  numbers  of  simulators  and  SAFOR  forces 
located  at  agreed  upon  test  points.  Once  the  operation  of  each  simulator  was  verified,  the  scope 
of  the  testing  was  expanded  to  include  mission  scenarios.  The  tests  did  not  thoroughly  stress  the 
system  because  the  availability  of  troops  £ind  facilities  was  limited. 

Recommendation ;  In  addition  to  the  necessary  technical  and  engineering  testing/ 
rehearsing,  time  must  he  scheduled  for  the  non-trainee  participants  to 
conduct  rehearsals  of  the  training  scenarios  and  variables. 

During  rehearsal,  a  full  load  should  be  placed  on  the  network  so  that  problems  can  be 
identified  and  resolved  prior  to  the  actual  exercise.  Scenario  events  that  will  strain  the 
capabilities  of  certain  simulators  need  to  be  rehearsed  and  changed  if  necessary  to  stay  within 
those  limitations.  For  the  MDT2-CAS  exercise,  this  meant  repositioning  simulator  start  points 
and  rearranging  the  flow  of  enemy  forces  into  those  simulators'  fields  of  view.  Additionally, 
0/Cs  and  support  personnel  must  rehearse  their  orchestration  of  AARs  (discussed  in  Section 
2.4.3). 


Recommendation;  A  permanent  team  of  qualified  personnel  for  multi-Service  DIS 
exercises  is  desirable  to  consistently  perform  O/C  tasks. 

Of  central  importance  to  any  training  exercise  are  the  0/Cs  who  control  the  exercise,  play 
the  roles  of  higher  headquarters,  manipulate  the  scenario  to  focus  on  key  training  objectives, 
assess  trainee  performance,  and  provide  feedback  through  debriefs  and  After- Action  Reviews. 
They  must  be  experienced  professionals  and  true  experts  in  the  areas  to  be  trained.  A  permanent 
cadre  would  be  ideal  but  may  not  be  practical  for  geographically  separate  sites. 

If  a  permanent  O/C  team  is  not  available,  then  exercise  planners  should  note  that  the 
required  SMEs  in  multi-Service  combat  operations  are  scarce  resources  with  many  competing 
demands.  They  can  be  found  in  Service  and  Joint  staffs,  schools,  and  training  centers,  as  well  as 
in  field  units.  They  must  be  scheduled  months  in  advance,  however,  because  their  availability 
limits  when  training  is  possible. 

Additionally,  the  appropriate  number  of  O/Cs  to  support  reliable  evaluation  of  trainee 
performance  should  be  determined,  and  the  levels  of  experience  and  background  level  should  be 
defined  for  each  O/C  position. 

Recommendation;  0/Cs  must  receive  training  and  familiarization  on  procedures  to 
be  followed  to  conduct  the  exercise  and  on  the  capabilities  and  limitations  of 
simulators  and  systems  at  their  own  and  other  sites. 

Recommendation;  At  least  2  weeks  in  advance  of  the  exercise,  O/Cs 
should  visit  the  training  sites  to  review  materials  and  plans. 

If  O/Cs  are  not  permanently  assigned  to  simulator  sites,  particular  emphasis  should  be 
placed  on  their  pre-exercise  training.  Site  visits  and  prebriefs  are  essential.  The  training  should 
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ensure  that  the  0/Cs  understand  their  roles  and  those  of  others  on  the  0/C  team,  both  during  and 
after  the  exercise  (during  AARs  and  debriefs),  and  that  they  are  fully  aware  of  the  goals  of  the 
training  exercise,  the  nature  of  the  scenario,  and  how  observation  or  performance  measurement 
tools  are  to  be  used.  0/Cs  from  each  site  should  travel  to  other  sites  to  become  familiar  with 
those  sites'  particular  simulator/system  nuances. 

To  the  fullest  extent  possible,  0/Cs  should  be  involved  in  developing  the  training 
exercise  and  performance  measurement  approaches.  If  this  is  not  possible,  additional  "formal" 
training  and  practice  in  these  areas  should  be  provided.  The  0/Cs  must  thoroughly  understand 
the  structure  of  the  AAR,  the  relationship  of  the  measurement  system  to  the  AAR,  and  the  timing 
and  nature  of  0/C  inputs  to  support  the  AAR. 

0/C  familiarization  with  the  strengths  and  weaknesses  of  the  various  simulators  and  the 
degree  of  interoperability  among  these  simulators  is  imperative.  0/Cs  should  particularly  be 
trained  on  the  facilities  available  to  them  (instructor/operator  stations,  network  status  boards, 
etc.)  that  support  the  observation  and  control  of  planning  and  combat.  They  need  to  know  the 
administrative  procedures  for  initiating  and  ending  exercises  and  for  managing  exercise  flow, 
including  procedures  to  be  implemented  when  nodes  drop  off  and  re-enter  the  exercise.  They 
should  be  familiar  with  any  site-specific  issues  that  enhance  or  degrade  evaluation  of 
performance  at  that  site. 

Most  importantly,  O/Cs  must  understand  the  key  training  objectives,  tasks,  conditions, 
and  standards  to  be  trained. 

Recommendation:  After  final  review  and  adjustments,  the  exercise  baseline  and  all 
supporting  personnel  and  functions  should  be  "frozen." 

The  training  environment— the  training  objectives,  scenarios,  simulations,  simulators, 
assessment  tools,  facilities,  and  all  personnel— should  be  reviewed  and  checked  to  verify  that  it 
can  do  the  job.  After  any  final  adjustments,  the  environment  should  be  frozen  to  avoid  changes 
that  compromise  training  execution  as  designed. 

3.3.4  Trainee  Preparation 

Recommendation:  At  least  1  month  in  advance,  trainees  should  be  provided  with 
materials  on  the  tactical  situation,  combat  intelligence,  and  the  necessary 
operations  orders  for  the  exercise. 

Recommendation:  On  Day  1  of  the  rotation,  the  trainees  should  be  briefed  and 
familiarized  with  the  simulator  operations.  Do  not  begin  the  exercise 
on  Day  1. 

Preparation  of  trainees  should  begin  about  a  month  before  the  exercise.  Provide  the 
participating  units  with  sufficient  materials  so  they  can  do  rehearsals  of  the  anticipated  missions. 
At  a  minimum,  supply  them  with  a  description  of  the  tactical  situations  and  higher  order 
operation  orders. 
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In  preparing  for  training,  the  key  objectives,  mission,  exercise  scenario,  enemy 
characteristics,  and  resources  available  should  be  described  to  the  trainees.  Trainees  need  to 
understand  the  task  requirements  for  performance  at  their  simulator  and  site,  and  the  roles/tasks 
of  other  participants  with  whom  they  may  interact.  Key,  service-specific  language  should  be 
explained. 

In  addition,  it  is  critical  to  tell  the  trainees  how  simulator  and  networking  limitations  will 
affect  play  of  the  missions  and  trainee  participation.  Trainees  will  benefit  more  from  the  rotation 
if  they  start  with  realistic  expectations.  Time  should  be  allowed  on  Day  1  for  trainees  to  become 
familiar  with  their  simulators  and  with  the  differences  between  the  simulator  and  the  actual 
equipment.  A  written  checklist  of  features  to  be  explained  should  be  developed.  MDT2-CAS 
trainees  reported  that  they  were  not  prepared  to  begin  mission  training  exercises  on  the  first  day. 


4.0  ARCHIVE  EXERCISE 
4.1  Archive  Baseline 


Recommendation;  The  exercise  baseline  should  be  preserved  as  a  reference  for 
repeating  it,  reviewing  it,  or  using  it  as  a  model  for  developing  other 
distributed  training. 

The  baseline  record  of  an  exercise  should  include  training  objectives,  scenarios,  hardware 
configurations,  software  loads,  network  configurations  and  loads,  measurement  tools,  and  all 
participant  and  trainee  requirements. 

4.2  Archive  Performance  Data 


Recommendation;  AARs  and  electronic  records  should  be  maintained 
to  aid  in  deriving  "lessons  learned"  from  multi-Service  training. 

Standard  items,  such  as  operations  orders  and  planning  documents,  should  be  retained  in 
the  archive.  In  addition,  it  should  include  recordings— which  are  possible  using  DIS  technology 
and  video  cassettes  —  of  (a)  the  combat  and  communication  portions  of  the  exercise  and 
(b)  multi-Service  AARs.  Finally,  the  archive  should  contain  summaries  of  performance 
assessments. 

5.0  POST-EXERCISE  TRAINING  REVIEW 

5.1  Purpose  of  After-Training  Review  (ATR) 

Recommendation;  At  the  end  of  all  exercises,  an  After-Training  Review  (ATR) 
should  be  conducted  to  answer  the  following  major  questions; 


27 


(1)  Did  the  training  add  value  to  the  multi-Service  training  pipeline  and 
meet  the  needs  of  all  participating  troops? 

This  question  is  to  routinely  assure  training  quality  control.  It  is  a  "temperature"  check,  a 
"first-aid"  diagnosis  of  DIS  training  system  value  and  problems.  It  has  five  dimensions:  need  for 
the  training  multi-Service  DIS  can  provide,  credibility  of  the  simulation,  multi-Service  value, 
role  in  the  training  cycle,  and  the  expected  impact  of  the  training. 

(2)  Did  the  training  program  work  as  expected?  If  not,  what  needs  to  he 
Dxed,  added,  or  eliminated? 

This  question  can  be  asked  in  many  ways.  One  way  is  to  ask  how  well  the  training 
objectives  for  the  exercises  were  implemented.  To  what  extent  did  the  training  satisfy  those 
objectives?  Critical  or  negative  answers  can  point  to  limitations  in  simulator  capability  or 
deficiencies  in  scenario  design. 

(3)  How  useful  were  the  training  and  evaluation  tools  and  methods,  such 
as  orientations,  exercise  design,  AAR  methods? 

The  MDT2  approach  emphasizes  training  tools  and  methods  to  support  front-end 
preparation,  exercise  conduct,  and  AAR  for  multi-Service  training.  The  usefulness  of  these 
methods  needs  to  be  monitored  so  that  they  can  be  improved. 

Results  of  an  ATR  can  assist  the  participating  units  in  planning  follow-up  training  and 
future  use  of  DIS.  Longer  range,  the  ATR  data  can  be  archived  and  used  for  on-going  quality 
control  of  DIS  training.  For  example,  the  ATR  can  document  the  need  for  training  program 
upgrades  or  more  limited  changes  in  planning  and  managing  subsequent  DIS  rotations. 

5.2  Getting  Started 

5.2.1  Planning  an  ATR 

Recommendations:  The  following  procedures  should  be  used  in  planning  an 

After-Training  Review: 

•  Incorporate  ATR  surveys  and  interviews  into  plans  for  scheduled  DIS 
exercises. 

*  Include  an  ATR  specialist  as  part  of  the  team  for  planning  DIS 
rotations. 

’  Include  ATR  results  as  part  of  the  archival  information. 

These  recommendations  make  the  ATR  an  integral  part  of  an  exercise.  Planning  should 
start  early  and  be  part  of  the  baseline  plan  outlined  in  Section  2.1.  For  an  operational  training 
system,  the  ATR  methodology  should  provide  “lessons  learned”  to  archive  for  others  to  use. 
Recommendations  based  on  MDT2-CAS  are  oriented  more  toward  prototype  work. 
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5.2.2  Conductin2  an  ATR 


Recommendation:  Collect  biographical  information  from  each  player. 

Biographical  data  summarize  a  trainee’s  military  background  and  experience  including 
any  cross-training.  Data  forms  should  be  tailored  to  the  different  training  sites  and  Services.  The 
primary  purpose  of  biographical  information  is  to  help  organize  the  rest  of  the  ATR. 

Commanders  also  can  use  biographical  data  for  planning  future  training  and  assisting  career 
development. 

Recommendation:  Use  a  short,  training-value  questionnaire. 

This  questionnaire  provides  trainees  and  0/Cs  a  structured  method  to  record  whether  or 
not  their  training  needs  were  met  along  with  why  or  why  not.  It  should  be  given  after  some 
training  in  the  simulator  environment.  For  MDT2-CAS,  the  questionnaire  was  used  after  at  least 
two  simulated  battles  were  fought  and  AARs  completed.  The  results  are  useful  to  stimulate 
discussion  “back  home”  about  plans  for  future  training. 

Recommendation:  Administer  a  questionnaire  on  how  well  training  objectives  were 
satisfied. 

Judgments  by  participants  are  made  about  how  well  the  simulation  network  and  mission 
scenarios  satisfy  the  training  objectives.  0/Cs  and  commanders  should  select  the  highest  priority 
and  challenging  objectives  for  a  manageable  and  focused  assessment.  MDT2-CAS  emphasized  7 
out  of  25  objectives. 

Recommendation:  Conduct  group  debriefings  to  obtain  more  detailed  information 
about  what  worked,  and  what  did  not  work  in  the  training. 

After  training  ends,  participants  at  each  site  should  be  debriefed  to  (a)  check  whether 
significant  training  needs  were  or  were  not  met,  (b)  identify  problems,  and  (c)  recommend  ways 
to  improve  the  training  experience.  More  detailed  recommendations  based  on  initial  applications 
of  MDT2-CAS  training  are  provided  in  Appendix  F.  They  are  intended  as  models,  or  templates, 
not  prescriptions. 

5.3  Users  of  and  Uses  for  ATR  Data 


Recommendation:  After- Training  Review  data  should  be  collected  and  saved  for 

multiple  users  of  results,  including  unit  commanders,  training  managers,  and 
military  and  political  leadership.  The  data  also  should  be  forwarded  to  the 
Modeling  and  Simulation  Resource  Repository. 

Unit  commanders  can  use  ATR  results  to  communicate  with  subordinates  about  training 
needs.  They  also  can  use  the  data  to  determine  the  role  of  simulation  in  the  unit’s  training  cycle. 
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Training  managers  would  have,  in  the  ATR  data,  a  reference  source  of  information  for 
planning  and  running  future  exercises.  A  production  version  of  MDT2-CAS  would  be  served 
best  by  a  well-trained  and  experienced  cadre  for  multi-Service  training.  They  would  be 
responsible  for  adjusting,  revising,  and  updating  training  methods  and  tools,  and  also  for 
identifying  improved  ways  to  translate  training  objectives  into  simulation  exercises. 

The  military  and  civilian  leadership  will  need  continuing,  documented  evidence  of  the 
value  of  DIS-type  training  if  it  is  to  be  supported  at  the  levels  for  which  this  type  of  training 
inherently  needs  to  become  institutionalized.  Training  program  review  data  support  that  purpose. 

6.0  SUMMATION:  PLANNING  AND  CONDUCTING  A  DIS  EXERCISE 

Following  is  an  outline,  in  abbreviated  terms,  of  the  various  steps  recommended  by  the 
MDT2  development  team  to  be  taken,  from  inception  to  final  assessment,  in  planning  and 
conducting  a  DIS  training  exercise: 

•  Review  when  to  use  DIS  environment  for  tactical  training 

•  Assign  responsibilities 

•  Schedule: 

Participants 

Sites 

Network 

Observer/Controllers  (0/Cs) 

•  Identify: 

Training  objectives 

Who  is  being  trained  to  do  what? 

What  are  the  training  objectives  and  tasks? 

Keep  these  in  mind  throughout  design,  implementation,  and  conduct 
of  the  exercise 

Functional  requirements 

Scenarios  and  mission 

Assessment  metrics:  Measures  of  effectiveness/Measures  of  performance 
(MOE/MOP) 

Feedback/ After  Action  Review  for  multi-Service 
Requirements 

Who  gets  what  feedback  and  when? 

How  will  feedback  be  given? 

Service-specific  debriefs 

Exercise  preparation  and  execution 

Force  structure  and  environment 

Manned 

Computer  generated 
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Synthetic  environment 
Terrain  database 
Time  of  year/Time  of  day 
Special  conditions  (e.g.,  weather) 

Simulated  entities  (weapon  systems,  C2  nodes,  etc.) 

Exercise  management  requirements 
Security 
Network  status 
Mission  initialization 
Real-time  control 
Mission  planning 
After- Action  Review  requirements 

Communication  requirements 
Mission  execution 

Tactical  networks 
Command  and  control  networks 
Technical  monitoring 
Simulation 
Network 
Exercise  control 
Mission  planning 
After- Action  Review 
DIS  PDU  requirements 
Mission  execution 
MOE/MOP 
Exercise  control 
Mission  planning 
After- Action  Review  requirements 
Bandwidth  required 
Network  organization 

•  Review  site  capabilities: 

Modify  requirements 
Enhance  site  capabilities 

•  Freeze  requirements 

•  Develop  program  schedule: 

Development 
Functional  verification 
Exercise  conduct 

•  Conduct  site-specific  enhancements 

•  Verify  site  readiness 

•  Conduct  functional  verification: 

Entity  performance 
Weapon  effects 
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Terrain  registration 
Line  of  sight 
Communications 

Mission  execution 

Tactical  networks 
Command  and  control  networks 
Technical  monitoring 
Simulation 
Network 
Exercise  control 
Mission  planning 
After- Action  Review 
Data  logging 
MOE/MOP 

•  Finalize  exercise: 

Schedule 

Scenario 

Resources 

•  Freeze  exercise  baseline 

•  Publish  supporting  materials  (e.g.,  order  of  battle,  operations  orders,  simulator  limitations) 

•  Pretrain  and  rehearse  participants 

•  Finalize  administrative  and  logistical  details 

•  Conduct  exercise 

•  Archive  exercise  baseline  and  exercise  data 

•  Review  and  document  exercise  lessons  learned 
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LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


AAR 

After  Action  Review 

ACA 

Airspace  Coordination  Areas 

AGOS 

Air- Ground  Operations  School 

AL/HRA 

Armstrong  Laboratory/Aircrew  Training  Resources  Division 

ALO 

Air  Liaison  Officer 

ALSA 

Air,  Land,  Sea  Application  Center 

ARI 

U,S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 

ATR 

After  Training  Review 

BN 

Battalion 

CAS 

Close  Air  Support 

Cdr 

Commander 

Co 

Company 

DDR&E 

Director  Defense  Research  and  Engineering 

DFO/MULE 

Deployed  Forward  Observer/Modular  Universal  Laser  Equipment 

DIS 

Distributed  Interactive  Simulation 

DMDC 

Defense  Manpower  Data  Center 

DMSO 

Defense  Modeling  and  Simulation  Office 

DSInet 

Defense  Simulation  Internet 

EDT 

Eastern  Standard  Time 

FAC 

Forward  Air  Controller 

FSE 

Fire  Support  Element 

FSO 

Fire  Support  Officer 

FTX 

Field  Training  Exercise 

GMT 

Greenwich  Mean  Time 

IDA 

Institute  for  Defense  Analyses 

MCCDC 

Marine  Corps  Combat  Development  Command 

MDT2 

Multi-Service  Distributed  Training  Testbed 

MOE 

Measure  of  Effectiveness 

MOP 

Measure  of  Performance 

MST 

Mountain  Standard  Time 

MWTB 

Mounted  Warfare  Test  Bed 

NAWCAD 

Naval  Air  Warfare  Center  Aircraft  Division 

NAWCTSD 

Naval  Air  Warfare  Center  Training  Systems  Division 

NIU 

Network  Interface  Unit 

NTC 

National  Training  Center 

O/C 

Observer/Controller 

OUSD 

Office  of  the  Undersecretary  of  Defense 

PDU 

Protocol  Data  Unit 

PST 

Pacific  Standard  Time 

PVD 

Plan  View  Display 

S-2 

Staff  Intelligence  Officer 

S-3 

Staff  Operations  Officer 

SAFOR 

Semiautomated  Forces 

SEAD 

Suppression  of  Enemy  Air  Defense 

SIMNET 

SIMulator  NETworking 

SME 

Subject  Matter  Expert 

TAC-A 

Tactical  Air  Controller  -  Airborne 

TACP 

Tactical  Air  Control  Party 

TF 

Task  Force 

TOC 

Tactical  Operations  Center 

TTP 

Tactics,  Techniques,  and  Procedures 

UPAS 

Unit  Performance  Assessment  System 

VTC 

Video-Teleconference 

XO 

Executive  Officer 
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